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1.0  EXECUTIVE  SUMMARY 


This  report  describes  the  research  and  development  (R&D)  activity  accomplished  during  the 
execution  of  Warfighter  Interface  Research  and  Technology  Operations  (WIRTO)  Task  Order 
009.  This  effort  focused  upon  the  development  of  an  Image  Fusion  Demonstration  Program 
intended  to  illustrate  the  image  quality  associated  with  different  techniques  for  blending  and 
fusing  multi-sensor  imagery.  The  demo  system  consists  of  three  components:  (1)  an  OpenGL- 
based  approach  for  blending  multiple  image  types  and  combinations  for  a  given  image,  (2)  a  viewer 
for  comparing  OpenGL-based  blending  and  image  fusion  techniques  for  a  given  image  set,  and  (3)  a 
viewer  that  demonstrates  image  fusion  results  for  multiple  image  sets. 


l 


Distribution  A:  Approved  for  public  release;  distribution  unlimited 


2.0  SCOPE 


2.1  Introduction 

The  primary  work  was  developing  an  image  fusion  demonstration  capability.  The  work  took  two 
different  spectral  bands  of  the  same  image  and  fused  them  together  to  develop  a  single  enhanced 
image  of  the  scene.  This  effort  specifically  led  to  the  development  of  an  Image  Fusion 
Demonstration  Program  intended  to  illustrate  the  image  quality  associated  with  different 
techniques  for  blending  and  fusing  multi-sensor  imagery.  The  demo  system  consists  of  three 
components:  (1)  an  OpenGL-based  approach  for  blending  multiple  image  types  and 
combinations  for  a  given  image,  (2)  a  viewer  for  comparing  OpenGL-based  blending  and  image 
fusion  techniques  for  a  given  image  set,  and  (3)  a  viewer  that  demonstrates  image  fusion  results 
for  multiple  image  sets. 

2.2  Document  Overview 

This  document  outlines  the  technical  and  program  management  activities  and  accomplishments 
perfonned  in  developing  an  Image  Fusion  Demonstration  Program  that  showcases  image  fusion 
techniques  under  study  and  development  by  AFRL’s  Human  Effectiveness  Directorate, 
Warfighter  Interface  Division,  Battlespace  Visualization  Branch  (AFRL/RHCV). 
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3.0  SPECIFIC  PROJECT  ACCOMPLISHMENTS 

Technical  support  for  the  Image  Fusion  Demonstration  effort  involved  the  development  of 
several  software  programs  that  demonstrate  the  fusing  of  multiple  raw  images  into  a  single 
blended  or  fused  image.  These  programs,  built  using  OpenGL  blending  capabilities  and 
AFRL/RHCV-developed  image  fusion  capabilities,  included  an  Image  Blending  Viewer  demo, 
an  Image  Fusion  and  OpenGL  Blending  Comparison  demo,  and  an  Image  Fusion  demo. 
Technical  activities  performed  in  support  of  each  of  these  are  discussed  below.  This  is  followed 
by  a  brief  outline  of  program  management  activities  performed  in  support  of  the  task. 

3.1  Image  Blending  Viewer  Demo 

The  first  of  the  demos  to  be  developed  was  the  Image  Blending  Viewer  demo.  To  begin,  a  C++ 
Class  was  filled  out  with  various  OpenGL  functions  pertaining  to  blending.  Properties  supported 
by  the  class  include  GL  Blending  Function,  Depth  Testing  method,  Lighting  method  (LightO  and 
Lightl  are  currently  supported),  Material  Coloring  method,  and  Texture  Generation  mode.  All 
property  functions  developed  allow  for  setting  the  method  or  function  and  the  optional 
immediate  enabling  or  disabling  of  the  function  or  method.  Once  the  property  is  defined,  other 
OpenGL  API  functions  can  be  used  to  enable  or  disable  the  various  methods.  A  new  constructor 
was  defined  so  that  an  external  texture  mapping  holder  and  texture  count  can  be  passed  to  the 
class  by  reference.  This  can  support  any  number  of  textures  to  be  used  by  the  application  without 
the  need  for  providing  space  within  the  class  that  may  otherwise  be  wasted;  existing  constructors 
still  support  up  to  three  texture  maps. 

Making  extensive  use  of  the  OpenGL  wrapper  class  that  was  developed,  Ball  engineers 
developed  the  Image  Blending  Viewer.  As  depicted  in  Figure  1,  the  demo  displays  up  to  three 
registered  images  and  merges  them  using  an  OpenGL  blending  function.  This  figure  shows  a 
long-wavelength  infrared  (LWIR)  image,  a  medium-wavelength  infrared  (MWIR)  image,  and  an 
image  intensifier  tube  (IIT)  image  of  a  single  scene.  These  raw  images  are  at  the  periphery  of  the 
display.  The  center  image  in  the  figure  shows  the  result  of  blending  the  LWIR  and  MWIR 
images  using  the  OpenGL  blending  procedures.  The  image  types  to  be  blended  can  be  selected 
via  keyboard  command;  however,  a  timed  automatic  mode  of  the  program  serves  as  a  standalone 
demonstrator. 
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Figure  1.  Screenshot  from  Image  Blending  Viewer  Demo 

The  images  being  blended  are  shown  to  move  from  their  respective  positions  toward  the  center 
until  they  are  perfectly  superimposed.  After  a  short  delay,  the  next  permutation  of  the  fusion  is 
effected  with  two  (or  all  three)  of  the  other  images;  the  program  will  continuously  cycle  through 
all  of  the  combinations.  Text  displayed  under  each  image  indicates  each  image  type,  and  the 
blended  image  is  labeled  with  text  that  indicates  the  fusion  mode  being  demonstrated.  The  text  is 
displayed  using  the  textured  font  rendering  functions  previously  developed  for  the  in-house  C++ 
OpenGL  support  library. 
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3.2  Image  Fusion  and  OpenGL  Blending  Comparison  Demo 


The  second  demonstration  program  developed  under  this  effort  consisted  of  a  comparison  of 
OpenGL  blending  with  a  fusing  process  developed  by  AFRL/RHCV.  See  Figure  2.  The 
comparison  demo  displays  side-by-side,  a  fused  image  and  a  blended  image  of  the  same  two 
source  images.  As  shown  in  Figure  2,  the  program  displays  the  two  raw  images  in  the  top  corners 
of  the  display.  The  blended  and  fused  images  created  from  the  raw  images  are  shown  in  the 
center  of  the  display.  The  demo  program  works  with  a  given  image  set  name  that  is  hard  coded, 
but  it  can  be  made  to  work  with  any  number  of  LWIR  and  IIT  images  available  on  the  lab 
network  drive.  Note  that  the  greater  range  of  grayscale  values  in  the  fused  image  results  in  higher 
image  contrast.  Relative  to  the  blending  process,  the  fusion  process  also  retains  more  sharpness 
and  detail  in  the  image.  This  illustrates  an  advantage  of  AFRL/RHCV’s  fusion  process  over  the 
blending  process. 
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Figure  2  Screenshot  from  Image  Fusion  and  OpenGL  Blending  Comparison  Demo 
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3.3  Image  Fusion  Demo 

The  final  program,  shown  in  Figure  3,  was  the  Image  Fusion  Demonstration.  Components  of 
earlier  demos  were  re-worked  to  show  the  fused  images  instead  of  the  OpenGL  blended 
approximations.  In  scroll  mode,  the  individual  images  are  brought  together  using  OpenGL 
blending.  When  they  are  perfectly  superimposed,  the  program  displays  the  two  images  fused  in 
place  of  the  blended  image.  The  program  shows  only  LWIR  and  IIT  fused  images.  It  operates  in 
a  sequential  or  random  viewer  mode  within  a  selected  image  set.  Image  sets  can  be  selected  by 
use  of  the  Page  Up/Down  keys.  Three  different  image  sets  are  currently  available  for  use  with 
this  program.  The  LWIR,  IIT,  and  fused  versions  of  each  image  in  the  selected  image  sets  were 
converted  to  bitmap  (.bmp)  format  using  MATLAB  script  developed  for  this  effort.  The  file 
structure  of  the  BMP  images  was  consolidated  and  simplified  to  make  future  porting  of  the 
programs  easier. 
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Figure  3.  Screenshot  from  Image  Fusion  Demo 
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3.4  The  End  Product 

All  three  demonstration  programs  described  above  were  populated  with  help  menu  displays  that 
list  the  available  key  commands  and  their  function.  The  help  menus  are  displayed  when  the 
programs  have  been  idle  for  a  period  of  time,  and  they  can  also  be  accessed  by  pressing  a 
function  key.  This  facilitates  use  of  the  demonstration  programs  in  an  open  self-guided  tour 
environment.  Observers  are  aided  in  how  to  use  the  programs  and  are  encouraged  to  explore  and 
investigate  the  technology  on  display.  Status  messages  were  added  to  confirm  the  user’s 
requested  command  in-process  or  an  undefined  key  strike.  The  programs  share  a  similar  look  and 
employ  a  like  user  interface  design. 

As  the  project  came  to  an  end,  executable  files  for  the  three  demonstration  programs  described 
above  were  generated  on  a  CD.  A  working  file  structure  was  included,  allowing  the  programs  to 
be  run  directly  from  the  CD.  Alternately,  the  file  structure  can  be  copied  to  a  host  PC  hard  drive 
for  execution.  It  contains  all  image  bitmap  files,  font  files  used  by  the  programs,  and  the 
parameter  file  for  setting  program  parameters.  Project  development  files  and  source  files  for  all 
programs  and  support  libraries  and  classes  were  also  included  on  the  CD. 

The  demo  programs  were  briefed  to  the  Government  Technical  Monitor  (TM)  and  interested 
colleagues.  The  programs  were  accepted  by  and  delivered  to  the  TM  at  the  conclusion  of  the 
technical  effort. 

3.5  Program  Management 

In  support  of  this  effort,  Ball’s  program  management  team  performed  a  number  of  functions  to 
ensure  that  the  technical  effort  was  perfonned  and  managed  within  the  funding  scope  and 
schedule.  The  Program  Manager  (Mr.  Rich  Doerr)  and  supporting  management,  financial,  and 
administrative  personnel  developed  a  number  of  task-specific  Contract  Data  Requirements  List 
(CDRL)  items.  These  included  data  a  Monthly  Funds  and  Man-Hour  Expenditure  Report,  a 
Quarterly  Contract  Funds  Status  Report,  and  a  Monthly  Project  Status  Report.  Additional 
deliverable-related  activities  included  the  review  and  delivery  of  any  Presentation  Material,  and 
development,  review  and  delivery  of  the  project  Final  Report. 

In  addition  to  deliverable  development,  the  Program  Manager  performed  a  number  of  functions 
to  coordinate  the  technical  activities  over  the  course  of  the  effort.  These  included  holding 
periodic  meetings  with  the  TM,  the  Contracting  Officer’s  Technical  Representative  (COTR),  the 
Ball  technical  team,  and  the  subcontractors.  Program  management  activities  also  included  task 
planning,  resource  management,  and  subcontractor  management  activities  to  ensure  that  AFRL 
was  receiving  the  support  required  throughout  the  life  of  the  program. 
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4.0  CONCLUSIONS 


In  summary,  this  effort  led  to  the  successful  development  of  demonstrations  that  illustrate  the 
visual  effects  of  combining  multi-sensor,  same  scene  imagery  into  a  single  fused  or  blended 
image.  To  the  extent  that  a  fused  or  blended  image  contains  more  essential  information  than  any 
of  the  individual  image  inputs  upon  which  it  is  built,  it  can  facilitate  a  user’s  understanding  of  the 
state  of  the  world  that  is  captured  in  the  imagery.  These  demos  illustrate  the  visual  effects 
associated  with  blending  and  fusing  techniques,  the  performance  impacts  of  which  can  be 
assessed  in  future  empirical  studies. 
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LIST  OF  ACRONYMS,  ABBREVIATIONS,  AND  SYMBOLS 


This  acronym  list  contains  both  general  acronyms  and  program-specific  acronyms. 


Acronym 

Definition 

AFRL 

Air  Force  Research  Laboratory 

CDRL 

Contract  Data  Requirements  List 

COTR 

Contracting  Officer’s  Technical  Representative 

DTIC 

Defense  Technical  Information  Center 

NT 

Image  Intensifier  Tube 

LWIR 

Long  Wavelength  Infrared 

MWIR 

Medium  Wavelength  Infrared 

R&D 

Research  and  Development 

RHCV 

Human  Effectiveness  Directorate,  Warfighter  Interface  Division, 
Battlespace  Visualization  Branch 

SOW 

Statement  of  Work 

TM 

Technical  Monitor 

TO 

Task  Order 

WIRTO 

Warfighter  Interface  Research  and  Technology  Operations 
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